\subsection{Video Streaming}
\label{prestudy:identifyproblemandscenario:video}
Loss of data is inevitable when transmitting data through an unreliable data channel, and it is important to know the consequences of such loss of data. This section investigates the consequences of data loss in a video stream. The audio stream in the video is neglected.

When compressing video there are several encodings to choose from. However, the classical principle of video encoding is based on frames of different types. Frames can be either a complete picture or a predictive description of a picture. Predictive frames describes a frame as a function of preceding or succeeding frames. This is implemented by popular codecs such as H.261, H.263, MPEG-2 and the more recent MPEG-4, along with various other video compression mechanisms \cite{ngan2001video,richardson2003h}.

As described, video data can be divided into two kinds of frames: Independent frames (intra-frames or key-frames, usually denoted I-frames) where the full image is stored, and Prediction frames (denoted P-frames), where only the differences between preceding frames are stored. Prediction frames can also be made Bi-directional (denoted B-frames), containing the differences on both preceding and subsequent frames, thus minimising the data stored. A sequence of frames starting with an I-frame including all P- and B-frames until the next I-frame, is often referred to as a \ac{GOP}. The relationship between frames in a \ac{GOP} is illustrated in Figure \ref{fig:gop}.

\begin{figure}[h!]
\centering
	\begin{tikzpicture}[>=stealth',shorten >=1pt,auto, semithick]
		            
	\tikzstyle{every state}=[rectangle, fill=white,text=black,,minimum height=1.3cm, minimum width=2cm, node distance=3cm]

		\node[state] (I1) {I};
		\node[state, right of=I1] (P1) {P};
		\node[state, right of=P1] (P2) {P};
		\node[state, right of=P2] (B)  {B};
		\node[state, right of=B, draw=gray, text=gray]  (I2) {I};

		\path	(P1)	edge[->,bend right=75] node {}	(I1);
		\path	(P2)    edge[->,bend right=75] node {}	(P1);
		\path	(B)	edge[->,bend right=75] node {}	(P2);
		\path	(B)	edge[->,bend left=75] node {} (I2);
		
		\draw[decorate, decoration={brace,mirror,raise=10pt, amplitude=10pt}] (I1.south west) -- (B.south east) node[midway, below,yshift=-20pt] {Group of Pictures};
		
	\end{tikzpicture}
\caption{A \ac{GOP} of 4 frames, containing both I-, P- and B-frames. Depending on encoding, predictive frames may have multiple references to past or succeeding frames.}
\label{fig:gop}
\end{figure}

The I-frame is of higher importance than P- and B-frames because it represents a new reference point in the video. Any errors resulting from previously missing I-, P- or B-frames will be nulled upon a new I-frame. For error resilience, a video consisting only of independent frames would be optimal, but this conflicts with the data compression achieved by utilising the dependency in a video. The amount of independent frames that can be achieved in a video without affecting source data compression is investigated in Section \ref{subsec:gopandtotalfilesize} on Page \pageref{subsec:gopandtotalfilesize}.

The severity of losing a frame depends on the type of frame that is lost and the video context. An intuitive evaluation is given: If a scene is almost static, missing frames will probably not matter much. Loosing a predictive frame immediately before an I-frame will probably not be noticed either. Loosing predictive frames immediately after an I-frame would induce an error which would propagate until the next I-frame. Rapidly changing scenery will probably require all frames for proper quality.  The following prioritisation is given: The I-frame is the most important data, the predictive frames following immediately after an I-frame is the second most important data and finally, the least important data is the frames right before a new I-frame. Similar to the field of psycho-acoustics, evaluating the quality perception of a video, as experienced by the human eye, is non-trivial. The given data prioritisation will suffice for initial development and hopefully allow for a graceful quality degradation if not all video data can be received. 

The idea of graceful degradation of a video stream is not new. \ac{SVC} has been introduced to the H.264 standard. The general idea behind \ac{SVC} is a layered approach when encoding a video as illustrated in Figure \ref{fig:svcvideo}. A low quality video stream defines the base-layer and multiple enhancement layers provides additional information to increase video quality, such as improved resolution, frame rate or quantization \cite{ngan2001video}. Losing data on a video encoded with \ac{SVC} is more likely to cause a drop in resolution, frame rate or quantization, rather than a discontinuity in the video. Furthermore, \ac{SVC} allows computationally weak devices to decode only the lower layers of a high quality video stream. Server-side a single video encoded in \ac{SVC} suits a range of possible receivers, reducing storage requirements. \citep{RADVISION}   

\begin{figure}[h!]
\centering
	\begin{tikzpicture}[>=stealth',shorten >=1pt,auto, semithick]
		            
	\tikzstyle{every state}=[rectangle, fill=white,text=black,,minimum height=0.7cm, minimum width=10cm, node distance=1cm]

		\node[state] (BASE) {Base Layer (Conventional video coding)};
		\node[state, above of=BASE] (E1) {Enhancement Layer 1 (Framerate)};
		\node[state, above of=E1] (E2) {Enhancement Layer 2 (Resolution)};
		\node[state, above of=E2] (E3)  {Enhancement Layer 3 (Quantization)};

%		\path	(P1)	edge[->,bend right=75] node {}	(I1);
%		\path	(P2)    edge[->,bend right=75] node {}	(P1);
%		\path	(B)	edge[->,bend right=75] node {}	(P2);
%		\path	(B)	edge[->,bend left=75] node {} (I2);
		
%		\draw[decorate, decoration={brace,mirror,raise=10pt, amplitude=10pt}] (I1.south west) -- (B.south east) node[midway, below,yshift=-20pt] {Group of Pictures};
		
	\end{tikzpicture}
\caption{A rough model of the layer based structure of an SVC encoded video. The base layer must be available to the receiver, but the other layers are expendable.}
\label{fig:svcvideo}
\end{figure}

Only a very limited number of actual implementations of \ac{SVC} exists, with H.264/SVC being the standardised codec and commonly used \ac{SVC} codec \cite{ngan2001video}. H.264/SVC is an extension to the popular H.264 codec, and is used in at least one video conferencing application designed by RADVISION \textregistered\footnote{www.radvision.com} \cite{RADVISION}.

Although \ac{SVC} is more error resilient than conventional video coding, it contains data that can be regarded as more important, namely the base layer. Increased protection of the base layer will ensure at least a low quality video to be played under extreme channel conditions. This has led RADVISION \textregistered\ to employ \ac{UEP} for their \ac{SVC} solution, presumably by Reed-Solomon codes \citep{RADVISION}. 

As discussed, \ac{UEP} definitely has a place in \ac{SVC} and perhaps also in single layer framed video. A discussion of different error correcting codes now follow.








%% Old stuff

% Undersøges af hvordan en upålidelig data kanal (som broadcast) påvirker en videostream. Herunder skal der gøres rede for strukturen af en normal enkodet video (opbygningen i frames).
%Digital video can be divided in to two main forms; uncompressed and compressed. \\
%Using the uncompressed format the video will not lose any quality, however it is also the format that requires the most bandwidth. \\


% OLD:  I-frames and P-frames, where the I-frames is full picture frames and P-frames are only all the parts of the picture that has changed. This is the general functionality of popular codecs such as H.261, H.263, MPEG-2 and the more recent codec MPEG-4 \cite{ngan2001video,richardson2003h}. This feature enables a logical way of determining the important data, that need higher error protection. \\
%Making the I-frames of a higher importance than P-frames, enables nodes with a higher transmission loss to at least receive the I-frames. This is not an ideal restoration of the video, but it is significantly better than no frames at all. \\
%This implementation will give the digital video one of the features of analog video, hence when the number of transmission errors is increasing the video quality is decreasing as opposed to no video at all.
